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Abstract 

This  paper  compares  two  complementary  International  Classification  of  Diseases, 
Ninth  Revision,  Clinical  Modification  (ICD-9-CM)  code  classifications  for 
detection  of  adverse  events  (AEs)  at  the  hospital,  State,  and  national  levels.  The 
classifications  are  the  Agency  for  Healthcare  Research  and  Quality’s  Patient 
Safety  Indicators  2003,  June  release,  Version  2.1  (PS  Indicators)  and  the 
Utah/Missouri  Adverse  Event  Classification  2002,  Version  1  (AE  Classes).  First, 
the  paper  describes  similarities  and  differences  between  the  two  classifications, 
such  as  intended  purpose,  process  of  development,  grouping  of  ICD-9-CM  codes, 
specificity,  and  sensitivity.  Second,  it  compares  the  ways  each  classification 
categorizes  ICD-9-CM  codes  into  indicators  or  classes  of  potential  AEs.  Third, 
the  paper  presents  the  number  and  percentage  of  Utah  inpatient  discharges 
(UTIDs)  with  any  PS  Indicator  over  3  years  (2000,  2001,  and  2002)  and  compares 
the  percentage  of  UTIDs  by  PS  Indicator  to  published  values  derived  from  a 
national  database,  the  Healthcare  Cost  and  Utilization  Project  (HCUP) 

Nationwide  Inpatient  Sample  2000  (NIS).  UTIDs  have  significantly  higher  values 
for  five  PS  Indicators  and  significantly  lower  values  for  seven  PS  Indicators  than 
NIS.  Fourth,  the  paper  presents  the  annual  percentage  of  UTIDs  with  any  AE 
Class  code  for  2000,  2001,  and  2002,  which  shows  a  small  but  significant  increase 
over  this  3-year  period.  The  AE  Classes  are  more  sensitive;  they  detect  far  more 
potential  injuries  due  to  medical  care,  or  AEs,  but  may  include  more  false 
positives  than  the  PS  Indicators.  The  PS  Indicators  are  more  specific;  they  detect 
fewer  potential  AEs  but  may  include  fewer  false  positives  than  the  AE  Classes. 


Introduction 

We  expect  miracles  from  modern  medicine.  In  fact,  advances  in  medicine 
have  increased  life  expectancy  and  enhanced  quality  of  life.  People  who  would 
have  died  or  been  bedridden  by  cancer,  diabetes,  heart  disease,  or  other  diseases 
even  a  decade  ago  now  survive,  and  even  thrive,  thanks  to  new  medical  devices, 
procedures,  and  medications.  However,  the  very  care  that  is  intended  to  heal  can 
also  cause  hann.  Highly  publicized  cases,  such  as  a  teenage  girl  who  died  after 
lung  transplant  surgery  due  to  a  blood  type  error,1  serve  as  tragic  reminders. 
Unfortunately,  these  cases  are  only  the  tip  of  the  iceberg.  Frequently  cited  studies 
have  estimated  between  44,000  and  98,000  deaths  per  year  due  to  medical 
management  nationwide.2  The  Institute  of  Medicine  has  reported  findings  from 
approximately  30  works  published  in  the  1990s  substantiating  “serious  and 
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widespread  errors  in  health  care  delivery  that  resulted  in  frequent  avoidable 
injuries  to  patients.”  Such  incidents  and  publications  have  increased  awareness  of 
errors  in,  and  injuries  due  to,  medical  management  (AEs)  rather  than  the  patient’s 
underlying  disease  or  condition. 

AE  underreporting 

AEs  are  underdetected  and  underreported,  both  within  health  care 
organizations  and  externally.'  Interviews  in  19  States,  including  Utah,  indicated 
numerous  reasons  for  underreporting,  such  as  health  care  facilities  lacking 
internal  systems  to  identify  events,  uncertainty  about  reporting  requirements,  a 
culture  of  nonreporting,  a  lack  of  enforcement  at  the  State  level,  bureaucratic 
burden,  competition  and  market  share,  fear  of  publicity,  fear  of  liability,4  and  lack 
of  a  common  AE  taxonomy.5  These  findings  suggest  that  AE  reporting  systems 
may  be  most  effective  if  they  are  easy  to  use,  but  not  so  simple  that  the 
information  reported  is  of  limited  value.5 

Use  of  ICD-9-CM  codes  in  hospital  discharge 
data  for  AE  detection 

In  its  call  for  improved  understanding  of  patient  safety  epidemiology,  the 
Agency  for  Healthcare  Research  and  Quality  (AHRQ)  has  identified  hospital 
discharge  data  as  one  of  six  useful  sources  of  information  on  AEs.5  Hospital 
discharge  data  are  among  the  few  forms  of  data  used  nationwide.  Because  the 
International  Classification  of  Diseases,  Ninth  Revision,  Clinical  Modification 
(ICD-9-CM)  coding  system  is  widely  used  in  hospital  discharge  data,  it  can 
provide  an  accessible,  low  cost,  and  efficient  means  of  detecting  AEs  in  hospitals, 
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State  and  nationwide.  Because  it  is  already  in  use,  this  system  adds  few  new 
burdens  on  hospital  and  State  resources.6  As  retrospective  surveillance  and  case¬ 
finding  tools,  ICD-9-CM  codes  in  discharge  data  can  complement  other  detection 
and  reporting  methods.  Though  hospital  discharge  data  and  ICD-9-CM  codes 
have  substantial  limitations,  the  codes  provide  an  attractive  starting  place  to 

6  7 

improve  AE  detection.  ' 

Hospitals  already  use  ICD-9-CM  codes  in  discharge  data  for  their  own  patient 
safety  surveillance  systems.  For  example,  at  a  large  teaching  hospital  in  Utah, 
ICD-9-CM  codes  related  to  medical  device  codes  detected  more  medical  device- 
related  hazards  and  AEs  than  five  other  detection  methods,  including  computer 
flag  surveillance,  online  incident  reporting,  telemetry  checklists,  clinical 
engineering  database,  and  a  postdischarge  patient  survey.  Also,  ICD-9-CM  codes 
related  to  medical  devices  detected  different  kinds  of  device-related  hazards  and 
AEs  than  the  other  detection  methods.  Review  of  a  random-selection  sample  of 
patient  records  in  a  large  teaching  hospital  in  Utah  with  ICD-9-CM  device  codes 
(N  =  141)  revealed  that  72  percent  had  a  confirmed  medical  device  AE.  Other 
Utah  teaching  hospital  researchers  have  investigated  enhanced  case  findings 
based  on  ICD-9-CM  codes,  followed  by  retrospective  chart  review  using  explicit 
criteria  to  detect  adverse  drug  events  (ADEs),  or  adverse  events  related  to 
medications.  Initial  results  suggested  that  such  methods  can  roughly  double  the 
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total  number  of  ADEs  detected  relative  to  those  found  by  computer  flag 
surveillance.9  These  studies  give  compelling  evidence  of  the  effectiveness  of 
ICD-9-CM  codes  for  AE  detection.  However,  they  are  limited  to  specific  teaching 
hospitals  and  specific  kinds  of  AEs. 

Several  States  also  have  developed  methods  based  on  ICD-9-CM  codes  for 
statewide  surveillance  of  AEs  and  to  assist  hospitals  with  their  own  internal 
quality  improvement  of  patient  care.  These  classifications  differ  in  purpose  of  AE 
detection  as  well  as  scope  and  kind  of  AEs  they  detect.  For  example,  the  Florida 
Agency  for  Health  Care  Administration  released  a  one-time  statewide  study  in 
2000  on  drug-caused  illnesses,  such  as  allergies  and  side  effects,  that  was  based 
on  codes  associated  with  these  adverse  drug  effects.10  To  date,  the  Utah 
Department  of  Health  has  sent  five  quarterly  reports  that  contain  statewide 
numbers  and  rates  of  inpatient  discharges  with  at  least  one  ADE  or  misadventure 
and  related  infonnation  to  all  acute  care  hospitals  in  Utah.  The  Department  also 
has  posted  the  reports  on  its  Web  site.11  Additionally,  all  40  Utah  acute  care 
hospitals  received  their  own  hospital  level  numbers  and  rates  for  ADEs  and 
misadventures.  The  Department  uses  the  term  “misadventures”  to  include  obvious 
errors  of  or  injuries  due  to  medical  care,  such  as  accidental  punctures  or 
lacerations  and  foreign  bodies  accidentally  left  in  patient’s  body  during 
procedures,  wrong  surgery,  etc. 

Adverse  event  classes  of  ICD-9-CM  codes 

AHRQ-funded  patient  safety  reporting  demonstration  projects  in  Missouri, 
New  York,  Utah,  and  Wisconsin  have  developed  similar,  but  not  identical, 
classifications  for  several  categories  of  AEs.5  Wisconsin  uses  four  categories  of 
ICD-9-CM  codes  for  AEs  (drug,  device,  procedure,  and  radiation),  including 
principal  and  secondary  diagnosis  codes  among  all  Wisconsin  general  acute  care 
hospitals.6  One  of  New  York’s  AHRQ-funded  projects  focuses  on  three  specific 
kinds  of  AEs:  new  acute  pulmonary  embolism,  acute  myocardial  infarction,  and 
postoperative  wound  infection.5  Based  on  published  research,  research  in 
progress,  and  input  from  a  national  expert  panel,  the  Utah  and  Missouri  studies 
expanded  Rolfs  and  associates’  previous  AE  classification.  The  expanded 
version,  Utah/Missouri  Adverse  Event  Classification,  2002  Version  1  (AE 
Classes),  is  being  validated  by  medical  chart  reviews  in  Missouri  and  Utah  (N  = 
approximately  7,200  charts  per  State).  All  40  Utah  acute  care  hospitals  are 
participating  in  the  chart  review.  In  Missouri,  36  study  hospitals  (a  convenience 
sample  of  all  123  Missouri  acute  care  hospitals)  are  participating  in  the  chart 
review.  (See  Van  Tuinen,  Elder,  Link,  Li,  Song,  Pritchett,  “Surveillance  of 
Surgery-Related  Adverse  Events  in  Missouri:  Using  ICD-9-CM  Codes,”  in  this 
volume  for  more  details  about  the  Missouri  sample.)  The  Utah  study  emphasizes 
misadventures,  a  subset  of  two  of  the  surgery-related  AE  Classes,  (e.g.,  patient 
injuries  likely  to  be  due  to  medical  care,  such  as  foreign  object  accidentally  left  in 
patient’s  body),  and  ADEs  in  the  entire  patient  population;  the  Missouri  study 
emphasizes  a  larger  number  of  surgery-related  AEs  among  surgery  patients.  The 
States  have  provided  their  statewide  and  hospital  information  on  AEs  to  their 
hospitals.  Based  on  this  infonnation,  some  Utah  hospitals  have  requested 
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additional  patient-level  information  for  in-depth  examination  of  AEs  and  changed 
or  added  programs  to  reduce  risk  to  patients. 

AHRQ  patient  safety  indicators 

AHRQ  and  Stanford  University  have  developed  20  Patient  Safety  (PS) 
Indicators  based  on  ICD-9-CM  codes.14  These  indicators  are  based  on  published 
research,  input  from  a  national  expert  panel,  medical  coding  experts,  and 
empirical  results  from  analysis  of  Healthcare  Cost  and  Utilization  Project  (HCUP) 
data.  The  intended  purpose  of  the  PS  Indicators  is  to  detect  potential  AEs, 
prioritize  areas  of  action,  or  to  provide  a  starting  point  for  further  analysis  to 
reduce  preventable  errors  through  system  or  process  changes.  Users  have  reported 
that  the  PS  Indicators  have  demonstrated  high  specificity  and  low  sensitivity.15  In 
other  words,  the  PS  Indicators  capture  relatively  few  false  positives  (potential 
AEs  that  further  inquiry  showed  were  not  caused  by  medical  care,)  but  they 
probably  miss  a  considerable  portion  of  true  positives,  or  actual  AEs.  In  this 
paper,  “potential  AEs”  are  errors  or  injuries  detected  by  selected  ICD-9-CM  codes 
that  are  more  likely  than  not  to  be  due  to  medical  care,  such  as  accidental 
puncture  or  laceration  during  a  procedure  (998.2). 

Rationale  for  this  study 

In  summary,  several  teaching  hospitals,  States,  and  a  federal  agency  have 
developed  ICD-9-CM  code  classifications  as  accessible,  low  cost,  efficient 
methods  for  detection  of  potential  AEs  for  State  public  health  surveillance  and  to 
assist  their  hospitals  with  internal  patient  safety  improvement.  However,  AHRQ 
and  most  of  these  States  have  developed  their  own  ICD-9-CM  code 
classifications.  Their  classifications  differ  in  number  of  codes,  ways  the  codes  are 
categorized,  ranges  of  AEs  they  detect,  and  the  degree  of  verification  for  their 
ability  to  detect  actual  AEs.  To  date,  no  national  findings  are  available  for  any  of 
the  state-developed  classifications,  and  no  State  findings  have  been  published  on 
the  PS  Indicators.  This  paper  addresses  this  gap.  It  compares  one  of  the  State 
classifications,  the  Utah/Missouri  Adverse  Event  Classification,  2002  Version  1 
(AE  Classes),13  and  the  AHRQ  Quality  Indicators,  Patient  Safety  Indicators  2003, 
Version  2.1  (PS  Indicators).14 


Methods 

Data  Sources 

The  national  dataset  is  the  HCUP  National  Inpatient  Sample  (NIS),  which 
consists  of  approximately  36  million  discharges  from  986  nonfederal  United 
States  acute  care  hospitals  in  the  year  2000.  For  this  paper,  the  authors  have  used 
published  values  for  number  and  percentage  of  NIS  discharges  by  PS  Indicator.16 
The  Utah  dataset  (Utah  inpatient  discharges  or  UTIDs)  consists  of  approximately 
710,000  inpatient  discharges  from  all  40  nonfederal  Utah  acute  care  hospitals  in 
2000,  2001,  and  2002. 17 
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Analysis 

First,  this  paper  compares  the  features  of  the  two  classifications,  the  PS 
Indicators  and  the  AE  Classes.  Second,  it  compares  the  ways  each  classification 
categorizes  ICD-9-CM  codes  into  indicators  or  classes  of  potential  AEs.  Third, 
the  paper  presents  and  compares  the  number  and  percentage  of  inpatient 
discharges  by  PS  Indicator  for  UTIDs  and  NIS.  Fourth,  the  paper  presents  the 
number  and  percentage  of  inpatient  discharges  by  AE  Class  for  UTIDs. 

Similarities  between  PS  Indicators  and  AE  Classes 

The  two  classifications  have  followed  similar  development  processes  and 
share  several  similarities.  Both  use  ICD-9-CM  codes  in  hospital  inpatient 
discharge  data  to  identify  potential  AEs.  Input  from  national  expert  panels  guided 
development  of  both  classifications.  Both  exclude  principal  diagnosis  codes  from 
the  numerator  for  each  indicator  or  class  to  focus  on  in-hospital  AEs,  rather  than 
AEs  that  originated  before  the  patient’s  hospital  stay.  Both  classifications  restrict 
the  denominator  of  some  indicators  or  classes  to  at-risk  patient  populations  (e.g., 
only  surgery  patients’  discharges  were  included  in  the  denominator  for  potential 
AEs  related  to  surgical  procedures). 

Differences  between  the  two  classifications 

Closer  inspection  reveals  several  differences  (Table  1).  First,  each 
classification  has  a  different  emphasis.  The  PS  Indicators,  which  target  events 
with  a  high  likelihood  of  representing  errors  in  medical  care,  are  intended 
primarily  for  institutional  case-finding  and  patient  safety  improvement 
initiatives.14  The  AE  Classes  are  intended  primarily  for  statewide  public  health 
surveillance,  though  hospitals  have  been  encouraged  to  use  State  and  hospital 
information  based  on  the  AE  Classes  for  internal  improvement  of  patient  care. 
Second,  the  PS  Indicator  expert  panel  consists  of  physicians  who  reviewed  PS 
Indicator  codes  prescreened  by  medical  records  coders,14  whereas  the  panel  for 
the  AE  Classes  includes  pharmacists,  nurses,  medical  records  experts,  as  well  as 
physicians,  all  of  whom  reviewed  each  AE  Class  code.  Third,  PS  Indicators  were 
developed  based  on  previous  findings.14  The  development  of  the  AE  Classes 
included  chart  reviews,  currently  in  progress,  in  Utah  and  Missouri  (N  = 
approximately  7,200  charts  per  State.) 

Fourth,  the  classifications  differ  in  scope.  The  PS  Indicators  consist  of  143 
ICD-9-CM  diagnosis  codes  and  codes  for  external  causes  of  injury  and  poisoning 
(E-codes)  in  three  categories:  three  obstetric  or  birth  trauma  indicators,  eight 
medical  indicators,  and  nine  surgery-related  indicators,14  but  none  specifically  for 
ADEs.  The  AE  Classes  consist  of  1,003  codes  in  3  categories  of  64  classes:  26 
ADE  classes,  22  medical  classes,  and  16  surgery-related  classes  (including  two 
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misadventure  classes).  The  Utah  study  has  emphasized  the  ADE  classes, 
because  the  Harvard  Study  reported  that  they  are  the  most  frequent  cause  of 
iatrogenic  injury.18 
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The  classifications  also  differ  in  how  they  group  the  ICD-9-CM  codes.  For 
example,  PS  Indicator  complications  of  anesthesia  (18  codes)  is  an  injury  “topic” 
(anesthesia,)  whereas  AE  Class  adverse  effects  of  anesthetics  and  other  central 
nervous  system  (CNS)  agents  (17  codes)  names  a  kind  of  injury  (by  certain  kinds 
of  drugs.)  Differences  in  the  grouping  of  codes  vary  by  Indicator/Class.  For 
example,  the  PS  Indicator  complications  of  anesthesia  includes  some  (but  not  all) 
of  the  codes  in  several  AE  Classes  (Table  2).  In  contrast,  the  PS  Indicator  and  the 
AE  Class  for  decubitus  ulcer  use  the  same  single  code,  790.7,  an  example  of  100 
percent  overlap  in  code  use  for  both  classifications.  However,  the  PS  Indicator  has 
more  inclusion  and  exclusion  criteria  than  the  AE  Class  for  decubitus  ulcer.  The 
PS  Indicator  includes  only  medical  and  surgery-related  discharges  and  only 
patients  with  a  length  of  stay  longer  than  4  days.  It  excludes  patients  with  any 
diagnosis  of  hemiplegia,  paraplegia,  or  quadriplegia.  The  AE  Class  includes  all 
patients  with  no  exclusions  except  principal  diagnosis  of  decubitus  ulcer.  The  PS 
Indicator  postoperative  sepsis  and  AE  Class  septicemia/bacteremia  are  an 
example  of  partial  overlap  in  code  use  that  is  less  complex  than  the  PS  Indicator 
complications  of  anesthesia.  The  PS  Indicator  and  the  AE  Class  use  the  same  13 
sepsis  codes,  but  the  fonner  also  includes  two  additional  codes  (995.91,  995.92 
systemic  inflammatory  response  syndrome  due  to  infectious  process  without  and 
with  organ  dysfunction,  respectively),  and  the  latter  also  includes  a  single 
bacteremia  code,  790.7.  The  PS  Indicator  postoperative  sepsis  includes  only 
elective  surgery  cases  or  a  patient  with  length  of  stay  longer  than  3  days,  and 
excludes  obstetric  patients  and  principal  diagnoses  of  infection  or  any  code  for 
immunocompromised  state  and  cancer.  AE  Class  sepsis  excludes  only  principal 
diagnosis  of  septicemia  or  bacteremia. 

Fifth,  definitions  of  numerators  and  denominators  for  the  PS  Indicators  and 
the  AE  Classes  influence  the  specificity  and  sensitivity  of  these  two 
classifications.  The  PS  Indicators  use  fewer  codes  in  the  numerator  and  more 
restricted  denominators  (patients  at  risk)  than  most  of  the  AE  Classes.  For 
example,  for  PS  Indicator  infection  due  to  medical  care,  only  two  infection  codes 
are  used  for  the  numerator,  and  patients  with  any  diagnosis  code  for 
immunocompromised  state  or  cancer  were  excluded  from  the  denominator.  The 
comparable  AE  Class  uses  more  codes  and  the  entire  inpatient  population  for  the 
denominator,  except  principal  diagnosis  for  infection  (see  Table  2).  In  fact,  users 
have  reported  that  the  PS  Indicators  have  demonstrated  excellent  specificity, 
although  their  sensitivity  is  quite  low,15  whereas  the  authors  of  this  paper  have 
found  the  AE  Classes  to  be  more  sensitive  and  less  specific.  In  other  words,  the 
PS  Indicators  may  miss  some  true  positives  (actual  AEs)  yet  include  fewer  false 
positives  (a  PS  Indicator  detects  a  potential  AE  but  further  investigation  shows  an 
actual  AE  has  not  occurred,)  while  the  AE  Classes  probably  capture  more  true 
positives  and  more  false  positives. 
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Class  47.  Other  medical  complications  not  specified. 

999.3  (1  code)  Other  infection  due  to  medical  care  999.3  is  1  of  10  codes  in  Class  47. 

not  elsewhere  specified 

Total:  2  codes  Total:  67  codes 


Table  2.  Comparison  of  AHRQ  Patient  Safety  Indicators  and  Utah/Missouri  Adverse  Event  Classes  for  complications  of 
anesthesia,  infection,  sepsis,  and  decubitus  ulcer,  cont. 
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Since  AHRQ  released  the  Patient  Safety  Indicators  (PSI)  software,14 
researchers  have  reported  national  rates  of  selected  PSI.16  Funded  by  AHRQ, 
several  large  health  systems  and  hospital  chains  are  exploring  methods  to  use  PSI 
for  internal  quality  improvements. 

Number  and  percentage  of  inpatient  discharges 
with  PS  Indicators  or  AE  Classes 

The  number  and  percentage  of  UTIDs  with  any  potential  AEs  detected  by  the 
PS  Indicators  were  determined  using  modified  versions  of  the  AHRQ  SAS 
programs.  For  each  PS  Indicator,  the  annual  number  of  UTIDs  was  small,  and 
differences  among  the  years  2000,  2001,  and  2002  were  not  statistically 
significant.  These  3  years  of  UTIDs  were  combined  to  calculate  the  percentage  of 
discharges  by  PS  Indicator.  The  authors  also  developed  SAS  programs  to  obtain 
the  annual  number  and  percentage  of  discharges  with  any  potential  AE  Class 
code.  The  AE  Classes  are  not  mutually  exclusive,  nor  are  the  PS  Indicators. 
Throughout  this  paper,  95  percent  confidence  intervals  (95%  Cl)  were  used  as  the 
criterion  for  statistical  significance  (alpha  =  0.05.) 


Results 

PS  Indicators  for  UTIDs  and  the  NIS 

The  number  and  percentage  of  UTIDs  with  any  PS  Indicator  are  22,298  of 
710,077  total  discharges  (3.141  percent).  The  published  number  and  percentage  of 
NIS  discharges  with  any  PSI  Indicator  are  1.07  million  of  36.32  million 
discharges16  (2.946  percent).  This  difference  is  statistically  significant  but  not 
substantially  different. 

At  the  individual  indicator  level,  the  differences  in  percentage  of  discharges 
for  UTIDs  and  NIS  are  statistically  significant  for  12  of  the  20  PS  Indicators 
(Table  3.)  Five  are  significantly  higher  for  UTIDs  than  for  NIS.  Of  particular 
interest  is  obstetric  (OB)  vaginal  trauma  with  instrument  (26.517  percent  for 
UTIDs  versus  24.408  percent  for  NIS),  which  has  the  highest  values  for  both 
samples.  Accidental  puncture/laceration  also  is  noteworthy,  because  the  UTIDs 
percentage  is  over  twice  the  NIS  percentage  (0.718  percent  for  UTIDs  versus 
0.324  percent  for  NIS).  Seven  differences  in  percentage  are  significantly  lower  for 
UTIDs  than  NIS.  Of  particular  interest  is  failure  to  rescue,  which  has  the  second 
highest  values  for  both  samples  (1 1.132  percent  for  UTIDs  versus  17.424  percent 
for  NIS),  and  OB  vaginal  trauma  without  instrument  (7.689  percent  for  UTIDs 
versus  8.659  percent  for  NIS),  which  has  the  third  highest  values  for  both 
samples.  Differences  for  the  remaining  eight  PS  Indicators  are  not  significant. 

AE  Classes  for  UTIDs  in  2000,  2001,  and  2002 

The  annual  number  and  percentage  of  UTIDs  with  any  AE  (Figure  1) 
increased  from  2000  through  2002  (53,351  and  23.07  percent  to  58,706  and  24.21 
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percent).  For  the  overall  percentage  for  any  kind  of  AE,  and  for  the  three 
categories  ofAE  classes — surgery-related  AE  (10,378,  16.91  percent  to  10,715, 
17.95  percent),  medical  AE  (44,374,  19.19  percent  to  49,121,  20.26  percent),  and 
ADE  (6,916,  2.99  percent  to  8,1 10,  3.34  percent) — the  increases  from  2001  to 
2002  and  from  2000  to  2002  were  significant,  but  no  increases  from  2000  to  2001 
were  significant.  For  subcategories  within  “any  ADE,”  increases  for  clinical  side 
effects  of  drugs  (1,659,  0.72  percent  to  1,983,  0.82  percent)  and  adverse  effects  of 
(a  specified  group  of)  drugs  (4,679,  2.02  percent  to  5,206,  2.15  percent)  mirrored 
significant  differences  for  “any  ADE.”  For  poisoning  by  (a  specified  group  of) 
drugs  (1,155,  0.50  percent  to  1,539,  0.63  percent,  the  increase  from  2000  to  2002 
is  significant.  All  other  increases  are  not  significant. 

Comparison  of  findings  from  the  PS  Indicators 
and  the  AE  Classes  for  UTIDs 

The  AE  Classes  detect  considerably  more  potential  adverse  effects  in  UTIDs 
than  the  PS  Indicators  (about  53,400  to  58,700  versus  about  7,400  to  7,600 
annually).  The  annual  percentage  of  UTIDs  with  any  AE  Class  is  more  than  seven 
times  that  for  the  PS  Indicators  (23.07  percent  to  24.21  percent  versus  3.14 
percent.) 


Discussion 

Overview:  differences  in  design,  purpose,  and  use 

The  higher  number  of  potential  AEs  detected  by  the  AE  Classes  was  expected, 
since  the  AE  Classes  use  more  ICD-9-CM  codes  (1,003)  than  the  PS  Indicators 
(143).  The  developers  of  the  AE  Classes  wanted  this  classification  to  be  inclusive 
in  the  early  stages  of  use  and  may  later  refine  the  classes  based  on  chart  review 
findings.  For  State  public  health  surveillance,  the  AE  Classes  detect  more 
potential  AEs  for  further  investigation  and  trend  analysis  over  time.  For  internal 
patient  care  quality  improvement  at  the  individual  hospital  level,  PS  Indicators 
can  help  hospital  personnel  identify  and  examine  “high  yield”  cases,  or  those  that 
are  most  likely  to  involve  actual  AEs. 

Differences  in  percentage  of  AEs 

There  could  be  several  explanations  for  statistically  significant  differences  in 
percentage  of  discharges  with  any  PS  Indicator  or  AE  Class.  First,  the  statistical 
differences  could  be  due  to  variations  in  coding  between  UTIDs  and  NIS.  On  the 
one  hand,  implementation  in  2001  of  Utah  patient  safety  administrative  rules, 
which  require  health  care  facilities  to  report  ADEs  using  their  ICD-9-CM  codes 
and  to  report  sentinel  events,  may  account  for  improved  reporting  of  some 
codes.19’ 20  For  example,  Utah  reports  more  E-codes  than  the  national  average. 
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Specifically,  an  AHRQ  internal  analysis  reveals  that  on  average,  in  2001,  the 
AHRQ  HCUP  database  contained  33.7  percent  of  medical  misadventure 
discharges  with  a  corresponding  misadventure  E-code;  but  UTIDs  reported  39.7 
percent  of  the  cases  (AHRQ  HCUP  internal  memo  describing  an  analysis  of  State 
E-code  reporting  and  accuracy,  dated  March  1,  2004).  On  the  other  hand, 
significantly  lower  UTIDs  values  for  some  of  the  PS  Indicators  may  mean  that 
Utah  hospitals  have  miscoded  or  underreported  the  conditions  these  PS  Indicators 
detect  compared  to  the  NIS. 

Second,  significant  differences  could  reflect  differences  in  actual  AEs,  which 
might  be  due  to  moderating  variables  in  Utah  compared  to  the  rest  of  the  Nation, 
such  as  variations  in  patient  characteristics,  medical  practice,  or  prenatal  care. 
Increases  for  AE  Classes  from  2000  to  2002  could  mean  that  that  rate  of  actual 
AEs  has  increased.  Improved  detection  would  be  “good  news,”  because  more 
detected  AEs  could  increase  understanding,  leading  to  their  reduction  and 
prevention.  More  actual  AEs  are  “bad  news,”  suggesting  more  harm  to  patients  is 
occurring. 

Limitations 

Though  statistically  significant  differences  alone  do  not  provide  definitive 
answers,  they  do  provide  a  useful  starting  point  for  both  health  care 
quality/patient  safety  improvement  and  statewide  surveillance.  Users,  such  as 
health  care  facilities  (e.g.,  hospitals)  and  public  health  institutions  (e.g.,  State 
departments  of  health),  can  start  with  these  differences  and  explore  possible 
explanations,  such  as  those  suggested  above. 

The  expert  panel  of  the  Utah/Missouri  Patient  Safety  Project  recognizes 
several  limitations  of  use  of  ICD-9-CM  codes  in  hospital  discharge  data  for 
detection  of  potential  AEs.  ICD-9-CM  codes  sometimes  are  incomplete.  Some 
health  care  facilities  and  public  health  institutions  do  not  capture  all  of  the  codes 
that  may  be  in  a  patient’s  discharge  records.  For  example,  currently  the  UTIDs 
database  has  nine  fields  for  ICD-9-CM  diagnosis  codes  plus  a  single  additional 
field  for  an  E-code.  Some  patients’  discharge  records  may  contain  additional 
codes  that  these  10  fields  do  not  capture.  Because  some  ICD-9-CM  E-codes, 
which  document  conditions  related  to  an  external  cause  of  injury,  are  not  directly 
related  to  reimbursement,  there  is  little  incentive  to  report  these  codes  at  this  time. 
Also,  only  conditions  documented  by  physicians  in  patients’  medical  records  are 
coded;  conditions  in  pharmacist,  nurse,  or  other  health  care  professional  notes  are 
not  coded. 

In  addition  to  possible  incomplete  recording  of  codes,  other  limitations 
include  the  inability  to  categorize  the  degree  of  harm  to  patients,  capture  all 
priority  AEs,  capture  near  misses,  and  provide  reliable  inter-institutional 
comparisons  due  to  coding  variations  among  facilities.  Additionally,  ICD-9-CM 
codes  in  discharge  data  do  not  distinguish  reliably  between  AEs  that  occurred 
during  hospitalization  (in-hospital  AEs)  from  AEs  that  occurred  before 
hospitalization  (arrived-with  AEs).  ’  Currently  codes  in  discharge  data  are 
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analyzed  after  the  patients’  hospitalizations  and  can  provide  retrospective,  but  not 
real-time,  information. 


Conclusion 

The  findings  of  this  paper  and  previous  works  suggest  that  no  “gold 
standards”  for  AE  detection  and  ICD-9-CM  AE  code  classification  currently 
exist.  While  some  detection  methods  appear  to  be  more  promising  than  others, 
optimal  tools  depend  on  the  user’s  purpose.  The  PS  Indicators  were  designed  with 
the  primary  purpose  of  helping  health  care  facilities  with  internal  patient  care 
quality  improvement.  Because  the  PS  Indicators  are  more  specific  and  probably 
include  fewer  false  positives,  they  provide  a  useful  starting  place,  a  subset  of 
“high-yield”  discharges  for  in-depth  examination  of  AEs.  The  AE  Classes  were 
designed  with  the  primary  purpose  of  statewide  surveillance  of  potential  AEs  and 
the  secondary  purpose  of  providing  facility-level  information  that  could  help 
facilities  with  their  own  internal  patient  care  quality  improvement  practices. 
Because  the  AE  Classes  are  more  sensitive,  they  detect  a  larger  number  of 
discharges  with  potential  AEs,  capturing  more  information  at  the  State 
surveillance  level,  providing  health  care  facilities  more  discharges  to  examine, 
and  addressing  the  patient  safety  issues  of  the  entire  inpatient  population. 

Possible  revisions  of  the  AE  Classes 

Development  of  improved  AE  detection  tools  based  on  ICD-9-CM  codes  still 
is  in  its  early  stages.  Further  improvements  and  changes  are  needed.  How 
indicators  or  classes  group  codes  is  one  design  consideration.  The  PS  Indicators 
group  codes  by  AE  topic,  such  as  complications  of  anesthesia,  whereas  the  AE 
Classes  group  codes  by  type  of  health  care  associated  injury  (or  AE),  such  as 
adverse  effects  of  anesthetics  and  other  CNS  agents.  Each  of  these  groupings  has 
advantages  and  disadvantages.  For  example,  the  PS  Indicator  for  complications  of 
anesthetics  tells  the  user  to  start  with  the  patients,  departments,  personnel, 
procedures,  policies,  medications,  and  equipment  involved  with  anesthesia;  but 
this  indicator  does  not  tell  the  user  whether  the  AE  involved  wrong  position  of  the 
endotracheal  tube,  malfunction  of  anesthesia  equipment,  or  the  anesthesia 
medication.  The  AE  Class  for  adverse  effects  of  anesthesia  and  other  CNS  agents 
tells  the  user  the  kind  of  injury  (or  AE),  but  this  particular  AE  Class  includes 
drugs  other  than  anesthetics.  To  examine  adverse  effects  of  anesthesia,  apart  from 
these  other  drugs,  the  user  would  need  to  access  anesthesia  codes  only. 

Possible  revisions  of  the  AE  Classes  could  include  changes  in  the  numerator 
or  denominator.  The  next  development  stage  for  the  AE  Classes  includes 
refinement  based  on  forthcoming  medical  chart  review  and  other  findings.  These 
refinements  may  include  dropping  existing  codes  that  identified  few  or  no  AEs 
compared  to  other  similar  codes  in  their  AE  Class,  and  adding  new  ICD-9-CM 
and  ICD-10  codes  that  identified  AEs  in  the  chart  reviews  that  are  not  already  in 
the  AE  Classes.  Currently,  both  the  PS  Indicators  and  the  AE  Classes  exclude 
principal  diagnosis  in  order  to  focus  on  in-hospital  AEs  in  contrast  to  arrived-with 
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AEs.  However,  inclusion  of  principal  diagnosis  could  be  useful  if  the  intention  is 
to  detect  AEs  in  a  larger  health  care  context  than  acute  care  hospitals  only.  For 
example,  a  patient  may  experience  an  AE  at  one  facility,  but  the  AE  may  be 
detected  later  during  a  subsequent  hospitalization  or  doctor’s  appointment. 
Examination  of  principal  diagnosis  codes  would  capture  at  least  some  of  these 
additional  cases  and  improve  the  detection  of  AEs  in  general.  Also,  as  mentioned 
earlier,  the  AE  Classes  denominators  have  fewer  restrictions  than  the  PS 
Indicators;  the  developers  are  considering  additional  restrictions  based  on 
forthcoming  chart  review  findings. 

Unit  of  analysis  is  another  consideration.  The  PS  Indicators  detect  the  number 
of  discharges  by  individual  indicator.  The  AE  Classes  can  detect  the  number  and 
rate  of  discharges  for  top-level  groupings  of  AEs — e.g.,  the  most  general  category 
(any  AE)  or  its  subcategories  (any  ADE,  any  surgery-related  AE) — as  well  as  at 
the  individual  AE  class  level  (e.g.,  adverse  effects  of  antibiotics  and  other  anti- 
infectives).  Other  possible  units  of  analysis  are  individual  AE  codes,  specific  AEs, 
and  patient  conditions.  For  example,  the  number  and  percentage  for  the  code 
E876.5  (performance  of  inappropriate  surgery)  would  be  useful  for  learning  more 
about  wrong-site  surgeries.  The  number  and  percentage  of  adverse  effects  of 
psychotropic  medication  for  a  particular  patient  or  group  of  patients  would 
provide  useful  infonnation  about  whether  a  particular  program  of  treatment  for 
mental  illness  is  effective,  as  would  “patient  condition,”  which  could  include 
readmission. 

The  findings  in  this  paper  are  based  on  retrospective  AE  detection.  As 
mentioned  earlier,  retrospective  and  real-time  detection  methods  can  detect 
different  AEs;  using  them  in  combination  could  improve  AE  detection. 
Improvements  in  current  methods,  such  as  computer  flag  surveillance,  and 
development  of  entirely  new  methods  are  possibilities.  Advances  in  medical  and 
health  knowledge  in  current  “gray  areas,”  such  as  pulmonary  embolism,  may 
clarify  the  distinction  between  disease  processes  and  AEs.  New  medical 
treatments,  medications,  and  devices  may  expose  patients  to  new  health  care- 
related  risks,  such  as  the  recently  approved  drug  eluting  vascular  stents.  Existing 
and  new  AE  detection  systems  will  need  to  adapt  to  both  advances  in  medical 
knowledge  about  current  gray  areas  as  well  as  new  potential  risks  to  patients,  and 
provide  timely  infonnation  for  better  understanding  of  AEs  and  how  to  prevent 
them. 
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